Skip to content

refactor: Slim down the build image (part 2) - #41

Merged
antobinary merged 5 commits into
bigbluebutton:v4.0.x-releasefrom
antobinary:40-slim-build-image
Nov 5, 2026
Merged

antobinary merged 5 commits into
bigbluebutton:v4.0.x-releasefrom
antobinary:40-slim-build-image

Conversation

@antobinary

Copy link
Copy Markdown
Member

Stacked on #40; the diff shows both until #40 merges.

What this does

Changes how the remaining tools are installed, without changing what is installed. The image goes from 2.95 GB (#40) to 2.17 GB on disk and from 727 MB to 561 MB compressed; the current image is 5.45 GB and 1.46 GB. Every package built with it is identical to the current image's build (see Verification).

  • One apt layer with --no-install-recommends, cleaned in the same layer; manuals, docs, info pages and locales are excluded through dpkg (copyright files stay).
  • openjdk-21-jdk-headless instead of openjdk-21-jdk, which pulled the X11 and GTK libraries of the full JDK into a container that never opens a window.
  • Node from the official tarball instead of nvm, without the C headers (node-gyp fetches them on demand; no package in the build compiles an addon against them).
  • Go 1.27.3 without its test, doc, misc and api directories, still at /go with GOPATH=/tools/go because Go records both paths in every binary it builds.
  • sbt as the launcher only: the 150 MB repository preloaded in the zip is dropped, the launcher fetches the sbt version each project pins into the ivy cache that the builds mount and keep.
  • The dev libraries are the ones the FreeSWITCH modules in modules.conf need, plus systemd-dev (bbb-record-core's build.sh asks pkg-config for the systemd unit directory; the systemd package that used to provide the file is no longer pulled in) and the five packages Drop the parts of the build image that no package build uses #40 keeps for coturn, libks and the meta-package.
  • python3-pip stays for the mediasoup worker build of bbb-webrtc-sfu.

Remaining weight, for the record: the apt layer is about 915 MB (JDK headless 286 MB, gcc toolchain 268 MB, dev headers 133 MB, cmake and its libicu 80 MB, perl 56 MB), Go 270 MB, Node 150 MB, sbt launcher 10 MB, base 88 MB.

Verification

All 31 packages of v4.0.x-develop (0cec77deeb) plus the bigbluebutton meta-package built with build/setup.sh, five at a time, on this image and on the current image on the same 16-core host; debs compared on control fields, dpkg -c listing, file contents, ELF sets and ldd library sets.

Every package is identical apart from version strings, changelog.gz timestamps, the 0.0.<epoch>-SNAPSHOT jar names of the three Java packages, and the random mktemp -d build folder that bbb-livekit's own build.sh embeds in its two Go binaries (two builds on the same image differ the same way). The meta-package builds here with the one-line copyright fix proposed separately for build/packages-template/bigbluebutton/build.sh.

Round of 30 packages, 5 lanes wall time
current image 695 s
#40 image 691 s
this image 716 s

Two things that bit while getting here, kept as comments in the Dockerfile: deleting /usr/share/man outright makes the JDK's update-alternatives fail, so the empty man directories stay; and lsb_release and the Go paths matter to the packages (see #40).

Follow-ups, not in this PR

  • The same Dockerfile applies to v4.1.x-release and v3.0.x-release with their tool versions.
  • -trimpath in the Go package builds and LIBEV_OK=0 in bbb-coturn's build.sh would remove the two environment dependencies that Drop the parts of the build image that no package build uses #40 and this PR work around.
  • build/setup.sh in bigbluebutton/bigbluebutton always pulls the image named in .gitlab-ci.yml; a DOCKER_IMAGE override that skips the pull for a local image made this work testable.

The image is used by build/setup.sh in bigbluebutton/bigbluebutton to build
every Debian package. Checking what the build/packages-template/*/build.sh
scripts on v4.0.x-develop actually call shows that a large part of what the
Dockerfile installs is never used:

- Grails 8.0.0 and the Gradle 9.8.1 -all distribution (680 MB unpacked):
  nothing invokes grails or gradle. bbb-web builds with ./gradlew, which
  downloads its own Gradle into the mounted cache; the akka applications
  build with sbt.
- FreeSWITCH -dev libraries for modules that are not built. modules.conf in
  build/packages-template/bbb-freeswitch-core does not list mod_av, mod_vlc,
  mod_shout, mod_perl, mod_snmp, mod_memcache, mod_portaudio or mod_alsa, and
  configure runs with --disable-core-odbc-support --disable-core-pgsql-support.
  This drops the ffmpeg, VLC, LADSPA/OCaml, Perl, SNMP, memcached,
  PostgreSQL, ODBC, PortAudio, ALSA, X11, LAME, Vorbis, Ogg, a52, gdbm,
  Berkeley DB, GnuTLS and apr development packages, plus groff.
- Debian and Maven packaging helpers: fpm builds every package, so cdbs,
  debhelper, devscripts, equivs, maven-debian-helper and the Maven/Freemarker/
  Gson jars, python3-apt, python3-debian and subversion are unused.
- openssh-server, vim, apt-utils, checkinstall, lsb-release,
  apt-transport-https and software-properties-common: nothing in a build
  uses them; apt speaks https natively on noble and no PPA is added.

Kept on purpose: yasm (FreeSWITCH's bundled libvpx needs it), python3-pip
(the bbb-webrtc-sfu build compiles the mediasoup worker), fakeroot (sbt
debian:packageBin), libyaml-dev, and everything else.

Two things only worked by accident and are made explicit: unzip was pulled
in by a removed package and is needed for the sbt zip, bbb-freeswitch-sounds
and the bbb-presentation-video download; uuid-dev came in through
libapr1-dev and libks needs it.

"RUN mkdir tools" followed by "RUN cd tools" never took effect, so sbt and Go
were extracted into / and SBT_HOME pointed at an empty path. The tools now
land under /tools through WORKDIR, with the /usr/bin links unchanged. GOPATH
moves from /tools/go to /root/go, because /tools/go is now GOROOT and Go
rejects a GOPATH equal to GOROOT.

The image goes from 5.45 GB to 2.93 GB on disk. No tool version changes.
build/packages-template/bigbluebutton/build.sh writes a control file and
calls equivs-build, so equivs is not unused after all; it pulls debhelper
back in as a dependency. Every other removal stands: a search of all
build.sh scripts, .gitlab-ci.yml and the GitHub workflow for the binaries
of the removed packages finds no other use inside the image.
…present

libapr1-dev used to pull libsctp-dev into the image, and coturn's configure
compiles its SCTP transport whenever /usr/include/netinet/sctp.h exists. Without
the header the turnserver, turnadmin and turnutils binaries of bbb-coturn come
out smaller and without SCTP support, the only difference a full build of all
packages found between this image and the current one. Installing libsctp-dev
explicitly keeps bbb-coturn byte-for-byte equivalent; the proper home for it is
the apt-get line in build/packages-template/bbb-coturn/build.sh.
Two more things a full build of every package turned up:

- coturn's configure runs lsb_release -si and, when it answers "Ubuntu", keeps
  LIBEV_OK off as a workaround for send_data_from_ioa_socket_nbh(). Without
  lsb-release the probe is silent and turnserver and turnadmin get the other
  code path. lsb-release stays.

- Go records GOROOT and GOPATH in every binary it builds (the builds do not
  use -trimpath). The current image extracts Go into / and uses
  GOPATH=/tools/go, so bbb-graphql-middleware, bbb-webrtc-recorder and
  bbb-livekit change when the tools move. The two no-op "mkdir tools" and
  "cd tools" lines are still removed, but sbt and Go keep landing in / and
  GOPATH stays /tools/go. Relocating the tools belongs with the image that
  changes how things are installed.
…launcher

Changes how the remaining tools are installed without changing what is
installed. The image goes from 2.95 GB to 2.17 GB on disk and from 727 MB to
561 MB compressed (the current image is 5.45 GB and 1.46 GB).

- One apt layer with --no-install-recommends, cleaned in the same layer;
  manuals, docs, info pages and locales are excluded through dpkg, copyright
  files stay. The empty man directories stay too: update-alternatives in the
  JDK postinst fails without them.
- openjdk-21-jdk-headless instead of openjdk-21-jdk, which pulled the X11 and
  GTK libraries of the full JDK.
- Node from the official tarball instead of nvm, without the C headers
  (node-gyp fetches them on demand; no package build compiles an addon).
- Go 1.27.3 without test, doc, misc and api, still at /go with
  GOPATH=/tools/go: Go records both paths in every binary it builds.
- sbt as the launcher only; the repository preloaded in the zip (150 MB) is
  dropped, the launcher fetches the pinned sbt into the mounted ivy cache.
- The dev libraries the FreeSWITCH modules in modules.conf need, plus
  systemd-dev (bbb-record-core's build.sh asks pkg-config for the systemd unit
  directory, and the systemd package that used to provide systemd.pc is no
  longer pulled in), libsctp-dev and lsb-release for coturn, uuid-dev for
  libks, equivs for the meta-package, python3-pip for the mediasoup worker.

All 31 packages of v4.0.x-develop plus the bigbluebutton meta-package were
built with this image and with the current one on the same host and compared
deb by deb (control fields, file lists, contents, ELF sets, ldd): identical
apart from version strings, timestamps and the SNAPSHOT jar names.
@antobinary
antobinary merged commit d854266 into bigbluebutton:v4.0.x-release Nov 5, 2026
1 check passed
imdt-claudiop pushed a commit to imdt-claudiop/bigbluebutton that referenced this pull request Nov 5, 2026
In bigbluebutton/bbb-docker-build-packages#40 and bigbluebutton/bbb-docker-build-packages#41 I drastically reduced the size of the image used for BBB packages creation. This is the point where we actually start using it.
antobinary added a commit that referenced this pull request Nov 6, 2026
… tarball Node

Brings the v4.0.x-release Dockerfile from #40 and #41 to v4.1.x-release.
The two lines differ only in tool versions, and the build/packages-template
scripts in bigbluebutton/bigbluebutton are the same on v4.1.x-develop as on
v4.0.x-develop apart from nginx config files, so the image needs the same
toolchain. Tool versions do not change here: GO_VERSION and NODE_VERSION
stay at 1.27.0 and 24.20.0, and the next commit updates them.

What goes:

- Grails 7.0.12 and the Gradle 8.14.3 -all distribution: nothing invokes
  grails or gradle. bbb-web builds with ./gradlew, the akka applications
  with sbt.
- FreeSWITCH -dev libraries for modules that modules.conf does not list,
  the ODBC and PostgreSQL libraries whose core support configure disables,
  and bison, python3-dev, libyaml-dev, sqlite3 and opus-tools.
- cdbs, debhelper, devscripts, the Maven and Freemarker/Gson jars,
  python3-apt, python3-debian, subversion, openssh-server, vim, apt-utils,
  checkinstall, apt-transport-https, software-properties-common and nvm.
- Manuals, docs, info pages and locales (copyright files stay), the JDK's
  X11 and GTK libraries (openjdk-21-jdk-headless), Node's C headers, Go's
  test/doc/misc/api trees and the repository preloaded in the sbt zip.

What stays, because a package build needs it: equivs (the bigbluebutton
meta-package runs equivs-build), libsctp-dev and lsb-release (coturn's
configure enables SCTP and picks its Ubuntu code path from them), yasm
(FreeSWITCH's bundled libvpx), python3-pip (the mediasoup worker),
systemd-dev (bbb-record-core asks pkg-config for the unit directory),
uuid-dev (libks), unzip, and Go at /go with GOPATH=/tools/go, because Go
records both paths in every binary it builds.

On v4.0.x-release this took the image from 5.45 GB to 2.17 GB on disk and
from 1.46 GB to 561 MB compressed, and every package of v4.0.x-develop
plus the meta-package built with the new image came out identical to the
one built with the old image. The reasoning behind each removal is in the
commit messages of #40 and #41.
antobinary added a commit that referenced this pull request Nov 6, 2026
[4.1 ports] Slim the build image, update Go and Node (port of #40, #41 and #35 to v4.1.x-release)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant